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REAL PARTY IN INTEREST 

The real party in interest in this appeal is Sony Corporation, a Japanese 
corporation, with offices at 7-35 Kitashinagawa 6-Chome, Shinagawa-ku, Tokyo, Japan, to 
which appellant has assigned all interest in, to and under this application, by virtue of an 
assignment recorded on January 23, 2001 at Reel 012020, Frame 0396 of the assignment records 
of the Patent and Trademark Office. 



RELATED APPEALS AND INTERFERENCES 

Upon information and belief, the undersigned attorney does not believe that there 
is any appeal or interference which will directly affect, be directly affected by or have a bearing 
on the Board's decision in the pending appeal. 
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STATUS OF THE CLAIMS 

Claims 1-3, 6, 9, 11, 21, 25-27, 30, 33, 35, 45 and 49-52 were rejected under 35 
U.S.C. § 103(a) as being unpatentable over Trower et al. (U.S. Patent 5,983,190: hereinafter 
referred to as "Trower") in view of Yamamoto (U.S. Patent 5,923,337). Claims 4, 5, 12, 13, 15, 
17, 19, 22-24, 28, 29, 36, 37, 39, 41, 43 and 46-48 were rejected under 35 U.S.C. §103(a) as 
being unpatentable over Trower in view of Yamamoto, and in further view of Houser et al. (U.S. 
Patent 5,774,859; hereinafter referred to as "Houser"). Claims 7, 8, 20, 31, 32, and 44 were 
rejected under 35 U.S.C. §103(a) as being unpatentable over Trower in view of Yamamoto and 
in further view of Lumelsky et al. (U.S. Patent 6,081, 780; hereinafter referred to as "Lumelsky"). 
Claims 10 and 34 were rejected under 35 U.S.C. §103(a) as being unpatentable over Trower in 
view of Yamamoto, and further in view of Crow et al. (U.S. Patent 6,262,724; hereinafter 
referred to as "Crow"). Claims 14, 16, 18, 38, 40 and 42 were rejected under 35 U.S.C. §103(a) 
as being unpatentable over Trower in view of Yamamoto, in further view of Houser, and yet 
further in view of Florin et al. (U.S. Patent 5,583,560; hereinafter referred to as "Florin"). 
The status of the claims is as follows: 

Claims allowed: none 

Claims objected to: none 

Claims rejected: 1-52 
The rejected claims 1-52 are set out in the Appendix. 

Appellant is appealing the Final Rejection of claims 1-52, which constitute all of 
the currently pending claims in this application. 
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STATUS OF THE AMENDMENTS 

The last amendment filed in this case was an Amendment After Final Rejection 
Under 37 C.F.R. §1.116 filed by the Appellant on July 5, 2005. In the amendment, applicant 
argued against the rejection of the claims based on the cited references. Additionally, new 
claims 53-55 were added. This amendment, however, was not entered for the purposes of this 
appeal. 
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SUMMARY OF THE INVENTION 

The present invention relates to a system and method of supporting operations for 
input of user commands to household electric appliances, such a television set /monitor and 
information equipment.. In particular, the invention relates to an interactive operation support 
system and method that permit input of user commands to various types of connected equipment 
in an interactive manner. Specifically, the system and method of the present invention are 
adapted for input of user commands in a natural form through a personified assistant, by means 
of inputting speech in order to interact with a personified assistant. Specification, page 1, lines 
13-27. 

As explained in the Background section of the Specification, "there is recently provided 
an interactive operation support system, which is set to allow a personified assistant to appear on 
a display screen, permitting the user to perform input of commands to the equipment in the form 
of carrying out a conversation face to face with an assistant on the screen." Specification, page 
3, lines 10-15. "However, such a system making it possible to present the status conditions in 
progress related to such operations on the screen through an assistant has not been developed so 
far, and as a result, there is no other way other than the user having to wait for response from the 
system with one's eyes fixed on the screen. It is to be even supposed that if the user gives a 
command to the system to execute a processing requiring a response time, the user would even 
misunderstand that the equipment is out of order .... Thus, it is preferable that, in order to allow 
the user to operate the equipment based on interaction with the assistant, an operationally easy 
input of a command system produces an effect close to natural language is provided." 
Specification, page 4, lines 11-23. 
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In other words, there is a need for a system wherein upon the input of a speech 
command the system produce an effect, apparently coming from an personified assistant on the 
screen, that is as close to natural language as possible. In this manner the user can tell by the 
assistant's words or body language that the assistant is working on a response to a user 
command, so as to avoid suggesting by silence that the system is out of order or otherwise 
confuse the user. 

To address the above-described difficulties in producing a system that enables a 
personified assistant to communicate with a user in a manner as close to natural language as 
possible, embodiments of the present invention relate to an interactive operation support system 
and method, which are adapted for input of user commands to the equipment in a natural form 
through a personified assistant, and in particular, to an interactive operation support system and 
method, which permit input of user commands by means of interaction with a personified 
assistant on a speech input basis. Specification, page 1, lines 20-27. 
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THE ISSUES PRESENTED 

The following issues are presented in this appeal: 

1. Whether claims 1-3, 6, 9, 11, 21, 25-27, 30, 33, 35, 45 and 49-52 are 
unpatentable under 35 U.S.C. § 103(a) as being obvious over Trower in view 
of Yamamoto? 

2. Whether claims 4, 5, 12, 13, 15, 17, 19, 22-24, 28, 29, 36, 37, 39, 41, 43 and 
46-48 are unpatentable under 35 U.S.C. § 103(a) as being obvious over Trower 
in view of Yamamoto, and in further view of Houser? 

3. Whether claims 7, 8, 20, 31, 32, and 44 are unpatentable under 35 U.S.C. 

§ 103(a) as being obvious over Trower in view of Yamamoto and in further 
view of Lumelsky? 

4. Whether claims 10 and 34 are unpatentable under 35 U.S.C. § 103(a) as being 
obvious over Trower in view of Yamamoto, and further in view of Crow? 

5. Whether claims 14, 16, 18, 38, 40 and 42 are unpatentable under 35 U.S.C. 

§ 103(a) as being obvious over Trower in view of Yamamoto, in further view 
of Houser, and yet further in view of Florin? 

GROUPING OF THE CLAIMS 

It is the Appellant's intention that claims 1-52 stand or fall together. 
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ARGUMENTS 

Whether claims 1-3. 6. 9, 11, 21. 25-27, 30. 33. 35. 45 and 49-52 are unpatentable under 35 

U.S.C. § 103(a) as being obvious over Trower in view of Yamamoto? 

Claims 1-3, 6, 9, 11, 21, 25-27, 30, 33, 35, 45 and 49-52 were rejected under 35 

U.S.C. § 103(a) as being unpatentable over Trower in view of Yamamoto. The structure of 

system claim 1, as presented herein, recites an operation control unit that includes: 

"an assistant control means for generating a personified assistant 
and making said assistant appear on a screen of said display unit; 

an output speech control means for determining speech required 
for said assistant to output said assistant's speech to the outside 
through said speech output unit after speech synthesis; 

an input speech recognition means for recognizing user's voice as 
a speech inputted through said speech input unit; 

an interaction management means for managing interaction 
between said assistant and said user according to said assistant's 
speech determined by said output speech control means and said 
user speech recognized by said input speech recognition means; 

an ambient state generating means for enabling said personified 
assistant to make motions in a proper way or to act as if urging the 
user to input commands using the input speech recognition means 
when the interactive operating system is placed in a wait state : and 

a command interpreting means for specifying a user's intention or 
specifying said inputted user command based on a content of 
interaction traced by said interaction management means." 

(emphasis added) 

In describing the above-mentioned 103 rejection of independent claim 1, the 
Examiner acknowledged that Trower fails to teach or suggest the ambient state generating means 
limitation listed above, namely "generating an ambient state enabling a personified assistant to 
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make motions in a proper way or act as if urging the user to input, speech commands when an 
interactive system is in a waiting state." May 23, 2005 Final Office Action (hereinafter referred 
to as ''Office Action), page 3. Rather, the Examiner cited Yamamoto for teaching "such an 
ambient state generating means ( eyebrow raising, foot tapping, etc., Col. 14, Line 32- Col. 15, 
Line 3)." Office Action, page 3. It is respectfully submitted that Yamamoto as applied by the 
Examiner does not disclose the ambient state generating means of claim 1. 

The Specification of the present invention describes various kinds of operation 
support processing according to an interactive operation support system, which includes "the 
assistant making reactions based on speech synthesis and 3D-animation to the user interface." 
Specification, page 40, lines 3-7. As an example, when a television is powered on, a personified 
assistant named "Yoshio" appears on the screen and goes into an ambient state delightfully with 
such words as "Ah" or "Well." Specification, page 40, lines 17-24 and lines 28-29. It is further 
indicated, "[i]n this stage, a user speech to the effect that 'I want to watch television', when 
further provided to the assistant, is recognized and then interpreted as a command in the speech 
recognition unit 13. Then, the system 1 makes an affirmative answer such as 'OK' and 'All 
right' in a speech output form through 'Yoshio', and simultaneously, a program on the last 
channel (that is, a channel having been selected the last time the power was turned off) is 
projected in such a way as to be zoomed up." Specification, page 40, line 30 to page 41, line 7. 
That is to say, in one embodiment of the invention, upon powering on of the television system, a 
personified assistant, (e.g., Yoshio) will appear and go into an animated ambient state urging a 
response from a user, and upon such a user response or command, the system causes the assistant 
to answer said command in speech output form. 
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By contrast, Yamamoto discloses a method and system for communication 
between a performer and an audience through an animated character. As disclosed in 
Yamamoto, "the performer 121 is generally located in the performance booth 120, and the 
audience 122 is usually not aware of the presence. The performer 121 communicates with the 
audience 122 through the character 125. Although the audience cannot see the performer 121, 
the performer 121 can see and hear the audience through a monitor 132 and a head set 133 via an 

audio mixer 152 In this way, the performer 121 and the audience 122 interactively engage 

in a conversation ." Specification, col. 6, line 63 to col. 7, line 5 (emphasis added). In sum, 
Yamamoto discloses a performer interacting with an audience as a form of entertainment through 
an animated character, rather than disclosing a personified assistant interacting with a user to 
facilitate operation of a system, as claimed. 

Specifically, within its Specification, Yamamoto discloses generating an 
animation sequence of a character on a real-time basis that approximates human speech, in which 
the character is animated based on the performer's voice and other input. See Yamamoto, 
Abstract; col. 6, lines 45-62. In one embodiment, "[t]he performer 121 generally speaks into a 
microphone 136." Yamamoto, col. 6, lines 45-46. Later, "the voice input is processed bv a 
central processor 144 to determine a certain predetermined set of parameters for animating the 
character on the presentation display 124." Yamamoto, col 6, lines 49-51 (emphasis added). 
Further, "[i]n determining the animation, a controller 142 also polls additional input devices such 
as a control pad 138 and a foot switch 140. These input devices provides [sic] additional input 
signals for determining the animation sequence ." Yamamoto, col 6, lines 51-55 (emphasis 
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added). Finally, "[b]ased on the above-described input signals, the character 125 is animated in 
a lively manner." Yamamoto, col. 6, lines 61-62. 

That is to say, a character is animated in direct response to the voice and other 
input of a performer, and the character interacts with the audience through the actions of the 
performer. Yamamoto, col. 6, line 6 to col. 7, line 8. Therefore, Yamamoto fails to teach or 
suggest a personified assistant interacting with a user of "an interactive operation system when 
the system is placed in a wait state," as claimed. In addition, Yamamoto only discusses a 
performer causing a character to move or make motions, and thus fails to disclose an "an 
ambient state generating means enabling the assistant to makes motions in a proper way," as 
claimed. 

Moreover, Yamamoto discloses a personified character's motions that are directly 
animated by the performer's voice input or other actions. In effect, a performer's input causes 
the animated character to move and communicate with an audience (not with a user), in a similar 
way how a puppeteer would move a puppet's body to communicate with an audience. The 
character's actions do not prompt a response from a user. Yamamoto, col. 3, lines 28-41. 
Therefore, Yamamoto fails to teach or suggest " an ambient state generating means enabling the 
assistant . . . to act (e.g., by words or utterances) as if urging a response from a user " (emphasis 
added). Finally, Yamamoto only discusses inputting speech through a microphone, which is not 
described as having a speech recognition feature. Therefore, Yamamato fails to teach "urging 
the user to input commands using the input speech recognition means," as claimed. 

Thus, Yamamoto fails to teach or suggest an operation control unit that includes: 
an ambient state generating means for enabling a personified assistant to make motions in a 
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proper way or to act as if urging the user to input commands using the input speech recognition 
means when the interactive operating system is placed in a wait state , as recited in claim 1. As 
such, Yamamoto fails to achieve the invention's aim of simulating natural conversation between 
a user and assistant, in order to achieve improved operation support of an interactive operating 
system or apparatus. Therefore, it is maintained that Trower and Yamamoto, individually or in 
combination, fail to teach or suggest all the limitations of claim 1 . 

Accordingly, it is respectfully submitted that independent claim 1 is 
distinguishable from the applied or proper combination of Trower and Yamamoto for at least the 
reasons previously described. 

Since claims 25 and 49 closely parallel, and include substantially similar 
limitations as, claim 1, claims 25 and 49 are also believed to be distinguishable over the 
combination of Trower and Yamamoto for at least the reasons previously described. Claims 2-3, 
6, 9, 11, 26-27, 30, 33, and 35, 45 and 50-52 are dependent from one of the independent claims 1 
and 25, and due to such dependency, are also believed to be distinguishable from the applied or 
proper combination of Trower and Yamamoto for at least the reasons previously described. 

Whether claims 4. 5. 12. 13. 15. 17. 19. 22-24. 28, 29, 36, 37, 39, 41. 43 and 46-48 are 
unpatentable under 35 U.S.C. § 103(a) as being obvious over Trower in view of Yamamoto. and 

in further view of Houser? 
Claims 4, 5, 12, 13, 15, 17, 19, 22-24, 28, 29, 36, 37, 39, 41, 43 and 46-48 were 
rejected under 35 U.S.C. § 103(a) as being unpatentable over Trower in view of Yamamoto, and 
in further view of Houser. 
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Claims 4, 5, 12, 13, 15, 17, 19, 22-24, 28, 29, 36, 37, 39, 41, 43 and 46-48 are 
dependent from one of the independent claims 1 and 25, and due to such dependency, are also 
believed to be distinguishable from the applied or proper combination of Trower and Yamamoto 
for at least the reasons previously described. 

Regarding claims 4 and 28, the Examiner indicated that Houser recites the 
"[c]ontrol means determines speech for leading a user's intention when said command 
interpreting means fails to specify said user's intention or said inputted user command." 
Regarding claims 5 and 29, the Examiner indicated that Houser discloses the "[c]ommand 
interpreting means also interprets a command for controlling a function of said external 
apparatus including selection of a broadcast program channel and/or recording/reproducing of 
said broadcast program." Regarding claims 12 and 36, the Examiner indicated that Houser 
discloses "[i]n response to said command interpreting means interpreting an inputted user 
command as a channel selection command, control means shows a selected broadcasting 
program in a display window." Thus, Houser fails to teach or suggest: "an ambient state 
generating means for enabling a personified assistant to make motions in a proper way or to act 
as if urging the user to input commands using the input speech recognition means when the 
interactive operating system is placed in a wait state." This limitation is present in claims 4, 5, 
12, 28, 29 and 36, through their dependency on one of claims 1 and 25. 

Regarding claims 13 and 37, the Examiner indicated, "it would have been obvious 
to utilize a ring shaped menu for speech-enabled channel selection to provide an aesthetically 
pleasing user interface," even though this feature is not cited as being disclosed by the 
combination of Trower, Yamamoto and Houser. Regarding claims 15 and 39, the Examiner 

-13- 00318518 



PATENT 
450100-03355 

indicated, "it would have been obvious matter of design choice to utilize a menu shown in a 
binder for the display of program information," even though this feature was not cited as being 
disclosed by the combination of Trower, Yamamoto and Houser. Thus, Houser fails to teach or 
suggest: "an ambient state generating means for enabling a personified assistant to make 
motions in a proper way or to act as if urging the user to input commands using the input speech 
recognition means when the interactive operating system is placed in a wait state." This 
limitation is present in claims 13, 15, 37, and 39 through their dependency on one of claims 1 
and 25. 

Regarding claims 17 and 41, the Examiner indicated that Houser discloses "[i]n response 
to said command interpreting means interpreting an inputted user command as a channel change 
command, said assistant control means makes said assistant appear with a list of changeable 
broadcasting programs arranged in a matrix shape." Regarding claims 19 and 43, the Examiner 
indicated that Houser discloses, "EPG distributed as part of data broadcast is applied for 
generating a list of broadcasting programs in a matrix form." Regarding claims 22 and 46, the 
Examiner indicated that Houser discloses, "[communication means for connecting said system 
to a communication medium such as an external network or a telephone line, and a certifying 
means for certifying an information terminal connected to said system via said communication 
medium." Regarding claims 23, 24, 47 and 48, the Examiner indicated that Houser discloses, 
"the viewing guide that displays program information as applied to Claims 17 and 41 and as 
shown in Fig. 11." Thus, Houser fails to teach or suggest: "an ambient state generating means 
for enabling a personified assistant to make motions in a proper way or to act as if urging the 
user to input commands using the input speech recognition means when the interactive operating 
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system is placed in a wait state." This limitation is present in claims 17, 19, 22- 24, 41, 43, and 
46-48 through their dependency on one of claims 1 and 25. 

Accordingly, it is respectfully submitted that claims 4, 5, 12, 13, 15, 17, 19, 22- 
24, 28, 29, 36, 37, 39, 41, 43 and 46-48 are distinguishable from the applied or proper 
combination of Trower, Yamamoto, and Houser. 

Whether claims 7, 8, 20, 31. 32. and 44 are unpatentable under 35 U.S.C. § 103(a) as being 
obvious over Trower in view of Yamamoto and in further view of Lumelskv? 

Claims 7, 8, 20, 31, 32, and 44 were rejected under 35 U.S.C. §103(a) as being 
unpatentable over Trower in view of Yamamoto and in further view of Lumelsky. 

Claims 7, 8, 20, 31, 32, and 44 are dependent from one of the independent claims 
1 and 25, and due to such dependency, are also believed to be distinguishable from the applied or 
proper combination of Trower and Yamamoto for at least the reasons previously described. 

Regarding claims 7 and 31, the Examiner indicated that Lumelsky discloses, 
"communication means for connecting said system to a communication medium such as external 
network and/or a telephone line, and a mail exchange means for making an exchange of 
electronic mails via said communication medium, wherein said output speech control means 
determines said assistant's speech based on content of an incoming mail." Regarding claims 8 
and 32, the Examiner indicated that Lumelsky further discloses, "[interaction management 
means manages an input speech of a user as a message bound for another user, and said output 
speech control means determines said assistant's speech based on said message." Regarding 
claims 20 and 44, the Examiner recognized that the combination of Tower, Yamamoto, and 
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Lumelsky does not specifically suggest displaying a new or incoming mail icon in response to 
the acceptance of mail; however the Examiner took official notice that it is well known in the art 
to implement a visual means of representing a new e-mail so that a user can recognize new and 
unread e-mails. Thus, Lumelsky fails to teach or suggest: "an ambient state generating means 
for enabling a personified assistant to make motions in a proper way or to act as if urging the 
user to input commands using the input speech recognition means when the interactive operating 
system is placed in a wait state." This limitation is present in claims 7, 8, 20, 31, 32, and 44 
through their dependency on one of claims 1 and 25. 

Accordingly, it is respectfully submitted that claims 7, 8, 20, 31, 32, and 44 are 
distinguishable from the applied or proper combination of Trower, Yamamoto, and Lumelsky. 

Whether claims 10 and 34 are unpatentable under 35 U.S.C. §103(3) as being obvious over 
Trower in view of Yamamoto, and further in view of Crow? 
Claims 10 and 34 were rejected under 35 U.S.C. § 103(a) as being unpatentable 
over Trower in view of Yamamoto, and further in view of Crow. 

Claims 10 and 34 are dependent from one of the independent claims 1 and 25, and 
due to such dependency, are also believed to be distinguishable from the applied or proper 
combination of Trower and Yamamoto for at least the reasons previously described. 

Regarding claims 10 and 34, the Examiner indicated that Crow discloses that the 
"[assistant control means places said personified assistant in a room scattered with various kinds 
of objects including a link to an information resource, and in response to an interest of said user 
in a recording medium including a link to a music content placed in said room, said command 
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interpreting means interprets an inputted user command as a command for playing back said 
music content." 

Thus, Crow fails to teach or suggest: "an ambient state generating means for 
enabling a personified assistant to make motions in a proper way or to act as if urging the user to 
input commands using the input speech recognition means when the interactive operating system 
is placed in a wait state." This limitation is present in claims 10 and 34, through their 
dependency on one of claims 1 and 25. 

Accordingly, it is respectfully submitted that claims 10 and 34 are distinguishable 
from the applied or proper combination of Trower, Yamamoto, and Crow. 

Whether claims 14. 16, 18, 38. 40 and 42 are unpatentable under 35 U.S.C. § 103(a) as being 
obvious over Trower in view of Yamamoto, in further view of Houser, 
and vet further in view of Florin? 
Claims 14, 16, 18, 38, 40 and 42 were rejected under 35 U.S.C. §103(a) as being 
unpatentable over Trower in view of Yamamoto, in further view of Houser, and yet further in 
view of Florin. 

Claims 14, 16, 18, 38, 40 and 42 are dependent from one of the independent 
claims 1 and 25, and due to such dependency, are also believed to be distinguishable from the 
applied or proper combination of Trower and Yamamoto for at least the reasons previously 
described. 

The Examiner indicated that Trower in view of Yamamoto, and further in view of 
Houser, teaches the speech command interface system and method utilizing an interactive 
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animated character and featuring speech-initiated TV channel selection menus containing 
program information, as applied to Claims 13, 15, 17, 37, 39 and 41. As discussed above, the 
combination of Trower, Yamamoto, and Houser fails to teach or suggest the "ambient state 
generating means" limitation incorporated into claims 13, 15, 17, 37, 39 and 41, through their 
dependency on claims 1 or 25. Further, the Examiner recognized that these references do not 
teach the ability to zoom in on a selected channel, but asserted, "Florin discloses a picture-in- 
picture window with the ability to jump to (zoom in on) the channel displayed in the window." 

Thus, both Houser and Florin, individually or in combination, fail to teach or 
suggest: "an ambient state generating means for enabling a personified assistant to make 
motions in a proper way or to act as if urging the user to input commands using the input speech 
recognition means when the interactive operating system is placed in a wait state." This 
limitation is present in claims 14, 16, 18, 38, 40 and 42 through their dependency on one of 
claims 1 and 25. 

Accordingly, it is respectfully submitted that claims 14, 16, 18, 38, 40 and 42 are 
distinguishable from the applied or proper combination of Trower, Yamamoto, Houser and 
Florin. 

CONCLUSION 

Claims 1-3, 6, 9, 11, 21, 25-27, 30, 33, 35, 45 and 49-52 are not obvious in view 
of the applied or proper combination of Trower and Yamamoto. Accordingly, it is respectfully 
submitted that the Examiner erred in rejecting claims 1-3, 6, 9, 1 1, 21, 25-27, 30, 33, 35, 45 and 
49-52 and reversal of such rejections by this Honorable Board is solicited. 
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Claims 4, 5, 12, 13, 15, 17, 19, 22-24, 28, 29, 36, 37, 39, 41, 43 and 46-48 are not 



obvious in view of the applied or proper combination of Trower, Yamamoto, and Houser. 
Accordingly, it is respectfully submitted that the Examiner erred in rejecting claims 4, 5, 12, 13, 
15, 17, 19, 22-24, 28, 29, 36, 37, 39, 41, 43 and 46-48 and reversal of such rejections by this 
Honorable Board is solicited. 

Claims 7, 8, 20, 31, 32, and 44 are not obvious in view of the applied or proper 
combination of Trower, Yamamoto, and Lumelsky. Accordingly, it is respectfully submitted 
that the Examiner erred in rejecting claims 7, 8, 20, 31, 32, and 44 and reversal of such rejections 
by this Honorable Board is solicited. 



Trower, Yamamoto, and Crow. Accordingly, it is respectfully submitted that the Examiner erred 
in rejecting claims 10 and 34 and reversal of such rejections by this Honorable Board is solicited. 



combination of Trower, Yamamoto, Houser and Florin. Accordingly, it is respectfully submitted 
that the Examiner erred in rejecting claims 14, 16, 18, 38, 40 and 42 and reversal of such 
rejections by this Honorable Board is solicited. 



Claims 10 and 34 are not obvious in view of the applied or proper combination of 



Claims 14, 16, 18, 38, 40 and 42 are not obvious in view of the applied or proper 



Respectfully submitted, 



FROMMER LAWRENCE & HAUG LLP 




Reg. No. 25,506 
(212) 588-0800 
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APPENDIX 
CLAIMS ON APPEAL 

1. An interactive operation supporting system for an apparatus, comprising a display 
unit, a speech input unit, a speech output unit and an operation control unit, wherein said 
operation control unit includes: 

an assistant control means for generating a personified assistant and making said assistant 
appear on a screen of said display unit; 

an output speech control means for determining speech required for said assistant to 
output said assistant's speech to the outside through said speech output unit after speech 
synthesis; 

an input speech recognition means for recognizing user's voice as a speech inputted 
through said speech input unit; 

an interaction management means for managing interaction between said assistant and 
said user according to said assistant's speech determined by said output speech control means 
and said user speech recognized by said input speech recognition means; 

an ambient state generating means for enabling said personified assistant to make 
motions in a proper way or to act as if urging the user to input commands using the input speech 
recognition means when the interactive operating system is placed in a wait state; and 

a command interpreting means for specifying a user's intention or specifying said 
inputted user command based on a content of interaction traced by said interaction management 
means. 
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2. The interactive operation supporting system according to claim 1, wherein said 
assistant control means determines an animation of said assistant based on a content of 
interaction managed by said interaction management means and/or an inputted user command 
specified by said command interpreting means. 

3. The interactive operation supporting system according to claim 1, wherein said output 
speech control means determines an assistant's speech based on a content of interaction managed 
by said interaction management means and/or an inputted user command specified by said 
command interpreting means. 

4. The interactive operation supporting system according to claim 1, wherein said output 
speech control means determines an assistant's speech for leading a user's intention when said 
command interpreting means fails to specify said user's intention or said inputted user command. 

5. The interactive operation supporting system according to claim 1, further comprising 
a connection means for connecting an external apparatus such as a television set/monitor 

and/or a video deck to said system, wherein said command interpreting means also interprets a 
command for controlling a function of said external apparatus including selection of a broadcast 
program channel and/or recording/reproducing of said broadcast program. 

6. The interactive operation supporting system according to claim 1, further comprising 
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a communication means for connecting said system to a communication medium such as 
an external network or a telephone line, 

wherein said input speech recognition means recognizes audio data received via said 
communication medium. 

7. The interactive operation supporting system according to claim 1, further comprising 
a communication means for connecting said system to a communication medium such as 

an external network and/or a telephone line, and 

a mail exchange means for making an exchange of electronic mails via said 
communication medium, 

wherein said output speech control means determines said assistant's speech based on a 
content of an incoming mail. 

8. The interactive operation supporting system according to claim 1, wherein said 
interaction management means manages an input speech of a user as a message bound for 
another user, and said output speech control means determines said assistant's speech based on 
said message. 

9. The interactive operation supporting system according to claim 1, wherein said 
assistant control means places said personified assistant in a room scattered with various kinds of 
objects including a link to an information resource. 
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10. The interactive operation supporting system according to claim 1, wherein said 
assistant control means places said personified assistant in a room scattered with various kinds of 
objects including a link to an information resource, and in response to an interest of said user in a 
recording medium including a link to a music content placed in said room, said command 
interpreting means interprets an inputted user command as a command for playing back said 
music content. 

11. The interactive operation supporting system according to claim 1, wherein in 
response to said command interpreting means interpreting an inputted user command, said 
assistant control means makes said assistant appear on a screen of said display unit. 

12. The interactive operation supporting system according to claim 1, further comprising 
a connection means for connecting a television set/monitor to said system, and in 

response to said command interpreting means interpreting an inputted user command as a 
channel selection command, said assistant control means makes said assistant show a selected 
broadcasting program display window. 

13. The interactive operation supporting system according to claim 1, further comprising 
a connection means for connecting a television set/monitor to said system, wherein in 

response to said command interpreting means interpreting an inputted user command as a 
channel change command, said assistant control means placing a ring-shaped changeable 
broadcasting program display window around said assistant. 
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14. The interactive operation supporting system according to claim 13, wherein, in 
response to selection of a desired channel by moving said display window on said ring upon a 
channel change command from said user and said assistant control means also zooms up said 
selected broadcasting program display window. 

15. The interactive operation supporting system according to claim 1, further comprising 
a connection means for connecting a secondary storage device for storing and/or 

reproducing a broadcasting program content to said system, 

wherein in response to said command interpreting means interpreting an inputted user 
command as a recorded program reproduction command, said assistant control means makes said 
assistant show a binder showing a view of recorded broadcasting program contents. 

16. The interactive operation supporting system according to claim 15, wherein in 
response to a selection for reproducing a recorded broadcasting program content, said assistant 
control means zooms up said selected recorded broadcasting program content display window. 

17. The interactive operation supporting system according to claim 1, further comprising 
a connection means for connecting a television set/monitor to said system, 

wherein in response to said command interpreting means interpreting an inputted user 
command as a channel change command, said assistant control means makes said assistant 
appear with a list of changeable broadcasting programs arranged in a matrix shape. 
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18. The interactive operation supporting system according to claim 17, wherein in 
response to a selection of a desired channel, said assistant control means zooms up said selected 
broadcasting program display window. 

19. The interactive operation supporting system according to claim 17, wherein EPG 
distributed as part of data broadcast is applied for generating a list of broadcasting programs in a 
matrix form. 

20. The interactive operation supporting system according to claim 1, further comprising 
a connection means for connecting a television set/monitor to said system, 

a communication means for connecting said system to a communication medium such as 
an external network and/or a telephone line, and 

a mail exchange means for making exchange of electronic mails via said communication 
medium, 

wherein said assistant control means makes an incoming mail display image to appear on 
a screen of said display unit in response to acceptance of mail. 

21. The interactive operation supporting system according to claim 1, further comprising 
a text conversion means for converting a Japanese ideogram Kanji related to text data 

displayed on a screen of said display unit into a corresponding phonetic character Kana. 



-25- 



00318518 



PATENT 
450100-03355 

22. The interactive operation supporting system according to claim 1, further comprising 
a communication means for connecting said system to a communication medium such as 

an external network or a telephone line, and 

a certifying means for certifying an information terminal connected to said system via 
said communication medium. 

23. The interactive operation supporting system according to claim 1, further comprising 
a connection means for connecting a television set/monitor to said system, and 

an extraction means for extracting text information from a received broadcasting program 
content. 

24. The interactive operation supporting system according to claim 23, wherein text 
information extracted by said extraction means is superimposed on a content of another 
broadcasting program on said screen. 

25. A method for supporting interactive operation of an apparatus or other externally 
connected apparatus including a display unit, a speech input unit and a speech output unit, 
comprising: 

an assistant control step for generating a personified assistant and making said generated 
assistant appear on a screen of said display unit; 

an output speech control step for determining a speech required for the assistant to output 
the assistant's speech to the outside through said speech output unit after speech synthesis; 
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an input speech recognition step for recognizing a user's voice as a speech inputted 
through said speech input unit; 

an interaction management step for managing interaction between said assistant and said 
user according to said assistant's speech determined by said output speech control step and said 
user speech recognized by said input speech recognition step; 

an ambient state generating step for enabling said personified assistant to make motions 
in a proper way or to act as if urging the user to input commands using the input speech 
recognition step when the interactive operating system is placed in a wait state; and 

a command interpreting step for specifying said user's intention or specifying said 
inputted user command based on a content of interaction traced by said interaction management 
step. 

26. The method for supporting interactive operation according to claim 25, wherein said 
assistant control step determines an animation of said assistant based on a content of interaction 
managed by said interaction management step and/or an inputted user command specified by 
said command interpreting step. 

27. The method for supporting interactive operation according to claim 25, wherein said 
output speech control step determines an assistant's speech based on the content of interaction 
managed by said interaction management step and/or an inputted user command specified by 
said command interpreting step. 
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28. The method for supporting interactive operation according to claim 25, wherein said 
output speech control step determines an assistant's speech for leading a user's intention when 
said command interpreting step fails to specify said user's intention or said inputted user 
command. 

29. The method for supporting interactive operation according to claim 25, wherein said 
apparatus further includes a connection means for connecting an external apparatus such as a 
television set/monitor and/or a video deck to said apparatus, and said command interpreting step 
also interprets commands for controlling a function of said external apparatus including selection 
of a broadcast program and/or recording/reproducing of said broadcast program. 

30. The method for supporting interactive operation according to claim 25, wherein said 
apparatus further includes a communication means for connecting said apparatus to a 
communication medium such as an external network and/or a telephone line, and said input 
speech recognition step recognizes audio data received via said communication medium. 

31. The method for supporting interactive operation according to claim 25, wherein said 
apparatus further includes a communication means for connecting said apparatus to a 
communication medium such as an external network and/or a telephone line, and a mail 
exchange means for making an exchange of electronic mails via said communication medium, 
said output speech control step determining said assistant's speech based on a content of an 
incoming mail. 
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32. The method for supporting interactive operation according to claim 25, wherein said 
interaction management step manages an input speech of a user as a message bound for another 
user, and said output speech control step determines said assistant's speech based on said 
message. 

33. The method for supporting interactive operation according to claim 25, wherein said 
assistant control step places said personified assistant in a room scattered with various kinds of 
objects including a link to an information resource. 

34. The method for supporting interactive operation according to claim 25, wherein said 
assistant control step places said personified assistant in a room scattered with various kinds of 
objects including a link to an information resource, and in response an interest of said user in a 
recording medium including a link to a music content placed in said room, said command 
interpreting step interprets an inputted command as a command for playing back said music 
content. 

35. The method for supporting interactive operation according to claim 25, wherein in 
response to said command interpreting step interpreting an inputted user command, said assistant 
control step makes said assistant appear on a screen of said display unit. 
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36. The method for supporting interactive operation according to claim 25, wherein said 
apparatus further includes a connection means for connecting a television set/monitor to said 
apparatus, and in response to said command interpreting step interpreting an inputted user 
command as a channel select command, said assistant control step makes said assistant show a 
selected broadcasting program display window. 

37. The method for supporting interactive operation according to claim 25, wherein said 
apparatus further includes a connection means for connecting a television set/monitor to said 
apparatus, and in response to that said command interpreting step interpreting an inputted user 
command as a channel change command, said assistant control step places a changeable 
broadcasting program display window with a shape of a ring around said assistant. 

38. The method for supporting interactive operation according to claim 37, wherein, in 
response to selection of a desired channel by moving said display window on said ring upon a 
channel change command from said user and said assistant control step also zooms up said 
selected broadcasting program display window. 

39. The method for supporting interactive operation according to claim 25, wherein said 
apparatus further includes a connection means for connecting a secondary storage device for 
storing and/or reproducing a broadcasting program content to said apparatus, and in response to 
said command interpreting step interpreting an inputted user command as a recorded program 
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reproduction command, said assistant control step makes said assistant show a binder showing a 
view of recorded broadcasting program contents. 

40. The method for supporting interactive operation according to claim 39, wherein in 
response to a selection for reproducing a recorded broadcasting program content, said assistant 
control step zooms up said selected recorded broadcasting program content display window. 

41. The method for supporting interactive operation according to claim 25, wherein said 
apparatus further includes 

a connection means for connecting a television set/monitor to said apparatus, and in 
response to said command interpreting step interpreting an inputted user command as a channel 
change command, said assistant control step makes said assistant appear with a list of changeable 
broadcasting programs arranged in a matrix shape. 

42. The method for supporting interactive operation according to claim 41, wherein in 
response to a selection of a desired channel, said assistant control step zooms up said selected 
broadcasting program display window. 

43. The method for supporting interactive operation according to claim 41, wherein EPG 
distributed as part of data broadcast is applied for generating a list of broadcasting programs in a 
matrix form. 
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44. The method for supporting interactive operation according to claim 25, wherein said 
apparatus further includes a connection means for connecting a television set/monitor to said 
apparatus, a communication means for connecting said apparatus to a communication medium 
such as an external network and/or a telephone line, and a mail exchange means for making 
exchange of electronic mails via said communication medium, and said assistant control step 
makes an incoming mail display image to appear on a screen of said display unit in response to 
acceptance of mail. 

45. The method for supporting interactive operation according to claim 25, further 
comprising 

a text conversion step for converting a Japanese ideogram Kanji related to text data 
displayed on a screen of said display unit into a corresponding phonetic character Kana. 

46. The method for supporting interactive operation according to claim 25, further 
comprising 

a communication step for connecting said apparatus to a communication medium such as 
an external network or a telephone line, and 

a certifying step for certifying an information terminal connected to said apparatus via 
said communication medium. 



47. The method for supporting interactive operation according to claim 25, wherein said 
apparatus further includes 
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a connection means for connecting a television set/monitor to said apparatus, and 
an extraction step for extracting text information from a received broadcasting program 
content. 

48. The method for supporting interactive operation according to claim 47, wherein text 
information extracted by said extraction step is superimposed on a content of another 
broadcasting program on said screen. 

49. A storage medium storing a software program in computer-readable form, said 
software program describing an interactive operation support processing for execution on a 
computer system, said interactive operation support processing applied to an apparatus including 
a display unit, a speech input unit and a speech output unit for supporting input of a user 
command to said apparatus and/or another externally connected apparatus, wherein said software 
program includes: 

an assistant control step for generating a personified assistant to make said generated 
assistant appear on a screen of said display unit: 

an output speech control step for determining a speech required to said assistant for 
output of said assistant's speech to the outside through said speech output unit after speech 
synthesis; 

an input speech recognition step for recognizing a user's voice inputted through said 
speech input unit as speech; 
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an interaction management step for managing interaction between said user and said 
assistant according to said assistant's speech determined by said output speech control step and 
said user speech recognized by said input speech recognition step; 

an ambient state generating step for enabling said personified assistant to make motions 
in a proper way or to act as if urging the user to input commands using the input speech 
recognition step when the interactive operating system is placed in a wait state; and 

a command interpreting step for specifying said user's intention or said inputted user 
command based on a content of interaction traced by said interaction management step. 

50. The interactive operation supporting system according to claim 1, further comprising 
a code conversion means for converting a character related to text data according to a 

group or system of characters or codes displayed on said display unit into a corresponding 
character of another group or system of characters or codes. 

51. The method for supporting interactive operation according to claim 25, further 
comprising 

a code conversion step for converting a character related to text data according to a group 
or system of characters or codes displayed on said display unit into a corresponding character of 
another group or system of characters or codes. 



52. The interactive operation supporting system according to claim 1, wherein said 
motions or said act includes a leading question. 
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